Skip to content

🤖 avm2: Emit StrictArray for dense AS3 Arrays in AMF0 (close #16381)#23772

Closed
MavenRain wants to merge 6 commits into
ruffle-rs:masterfrom
MavenRain:fix/amf-strict-array-for-as3-array
Closed

🤖 avm2: Emit StrictArray for dense AS3 Arrays in AMF0 (close #16381)#23772
MavenRain wants to merge 6 commits into
ruffle-rs:masterfrom
MavenRain:fix/amf-strict-array-for-as3-array

Conversation

@MavenRain
Copy link
Copy Markdown

Description

Closes #16381.

Per Lord-McSweeney's 2025-07-14 diagnosis, Ruffle was serializing AS3 Array
arguments as AMF0 ECMAArray (marker 0x08, with all entries as named string
keys "0", "1", ...) instead of AMF0 StrictArray (marker 0x0A, indexed by
position). Flash decoders see the former as {0: "x", 1: "y"} (associative
object) and the latter as ["x", "y"] (indexed array), which broke real-world
Flash Remoting endpoints that round-trip AS3 arrays as call arguments (the G2C
Online site that motivated the issue).

The fix unifies the AMF0 and AMF3 dense/sparse split in serialize_value
(previously the AMF3 branch did this split, the AMF0 branch dumped everything
into the sparse list with a // TODO: is this right? comment). For AMF0,
dense-only arrays now emit StrictArray; mixed arrays keep ECMAArray semantics.
AMF3 path is unchanged.

Byte-level confirmation

For new Array("G2C Online") in AMF0:

Before: 08 00 00 00 01 00 01 30 02 00 0A G2C Online 00 00 09 (ECMAArray
with sparse "0" key)
After: 0A 00 00 00 01 02 00 0A G2C Online (StrictArray,
length 1)
Flash: StrictArray(ObjectId(-1), [String("G2C Online")]) (matches
After)

Test coverage

The existing netconnection_send_remote test exercises the AMF0 args-wrap path
(top-level StrictArray of arguments), which was already correct before this PR
and stays correct after. All 5 netconnection tests plus the broader
amf/array/bytearray suites (~120 tests) stay green.

What we don't yet exercise is an AS3 Array passed AS an argument to
NetConnection.call, which is the bug case. Adding a SWF fixture requires Flex
SDK to recompile Test.swf, which I don't have set up locally. Happy to
follow up with a Rust unit test in a separate PR if you'd like dedicated
coverage; the heaviest piece there is mocking an Activation for
serialize_value.

Verification

  - cargo fmt --all --check
  - cargo clippy -p ruffle_core --tests --lib
  - cargo test --package tests for netconnection_*, amf*, array, bytearray (all
  green)

@danielhjacobs
Copy link
Copy Markdown
Contributor

This does not fix that site in testing.

Left is Ruffle, right is Flash:

Screenshot From 2026-05-21 09-48-27

Further information for diagnosis and testing: If you open the Network tab on DevTools and refresh http://www.g2conline.org/, with a Flash-enabled browser or with Ruffle, you will see a POST request sent to http://www.g2conline.org/g2camf/

If you right-click the request and copy it as cURL, and paste it in a Notepad, you'll see a cURL command ending with --data-raw ...

As raw data in Flash, you'll see this:

\x00\x00\x00\x00\x00\x01\x00\x12AMFApp.getDataMaps\x00\x02/1\x00\x00\x00\x17\n\x00\x00\x00\x01\n\x00\x00\x00\x01\x02\x00\nG2C Online

As raw data in Ruffle, both with this PR and the current version, you'll see this:

\x00\x00\x00\x00\x00\x01\x00\x12AMFApp.getDataMaps\x00\x02/1\x00\x00\x00\x19\n\x00\x00\x00\x01\x03\x00\x010\x02\x00\nG2C Online\x00\x00\x09

@danielhjacobs
Copy link
Copy Markdown
Contributor

Oh, you only changed AVM2. That's an AVM1 animation.

@danielhjacobs danielhjacobs added A-avm2 Area: AVM2 (ActionScript 3) T-compat Type: Compatibility with Flash Player llm The PR contains mostly LLM-generated code amf Issues relating to AMF serialization/deserialization T-fix Type: Bug fix (in something that's supposed to work already) and removed T-compat Type: Compatibility with Flash Player labels May 21, 2026
@danielhjacobs
Copy link
Copy Markdown
Contributor

danielhjacobs commented May 21, 2026

I see you fixed the tests. The above comment still applies. I'm guessing you'll have to further change core/src/avm1/globals/shared_object.rs

@kjarosh
Copy link
Copy Markdown
Member

kjarosh commented May 21, 2026

This needs tests to be merged.

@MavenRain MavenRain force-pushed the fix/amf-strict-array-for-as3-array branch from 4bde3ca to 8b581c0 Compare May 21, 2026 18:32
@MavenRain
Copy link
Copy Markdown
Author

I just pushed a follow-up addressing @danielhjacobs's diagnosis that G2C Online is an AVM1 animation. My original commit only touched the AVM2 path; this commit mirrors the same fix on the AVM1 side.

AVM1's serialize previously routed every Value::Object (including AS1 Arrays) through new_lso → recursive_serialize, which wraps the result as AmfValue::Object with "0", "1", ... property keys. Top-level AS1 Array arguments to NetConnection.call therefore appeared as bare Object on the wire, not even ECMAArray, which is worse than the AVM2 symptom this PR's first commit fixed.

Changes:

  1. core/src/avm1/globals/shared_object.rs::serialize now detects
    NativeObject::Array and routes it through a new serialize_array helper that
    emits AmfValue::ECMAArray (matching what flash-lso's ObjWriter::array /
    ArrayWriter::commit emit for nested arrays in recursive_serialize).
  2. core/src/avm1/globals/netconnection.rs::call and
    core/src/avm1/globals/local_connection.rs::send now wrap each serialize(...)
    result with promote_dense_ecma_to_strict_array (the helper added in the
    prior commit on the AVM2 path), recursively converting dense ECMAArray nodes
    to StrictArray. Both top-level and arbitrarily-nested AS1 arrays are now
    correct on the wire.
  3. core/src/avm2.rs makes the amf submodule pub(crate) so AVM1 can import
    the helper. Sharing across AVMs reflects the fact that the promotion rule
    is a property of the NetConnection / LocalConnection wire format, not a
    property of either AVM version; willing to refactor into a neutral module if
    reviewers prefer.

Verified locally against the same byte pattern from the G2C cURL capture:
the args wrap now produces StrictArray([StrictArray([String("G2COnline")])])
instead of StrictArray([Object([{name:"0", value:String("G2COnline")}])]).
All net-related test suites (localconnection, encoding1,netconnection, sharedobject, amf, array, bytearray)
stay green.

@danielhjacobs danielhjacobs added A-core Area: Core player, where no other category fits and removed A-avm2 Area: AVM2 (ActionScript 3) labels May 21, 2026
@MavenRain MavenRain force-pushed the fix/amf-strict-array-for-as3-array branch from 8b581c0 to b8e7c4c Compare May 21, 2026 20:48
MavenRain added 3 commits May 22, 2026 08:20
…16381)

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…sites

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…/LocalConnection (ruffle-rs#16381)

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
@MavenRain MavenRain force-pushed the fix/amf-strict-array-for-as3-array branch from b8e7c4c to 6aa45b9 Compare May 22, 2026 15:20
@danielhjacobs
Copy link
Copy Markdown
Contributor

danielhjacobs commented May 22, 2026

Please add tests for this behavior. You can feel free to pull and modify the test I added in ff0ed8a for the AVM1 behavior.

Note, If you change the AS file, you can compile the SWF for the AVM1 test from the top-level tests directory of this repo by running cargo testutils compile avm1/netconnection_serialize_arrays. Then, you can run a local web server using python tests/tests/swfs/avm1/netconnection_serialize_arrays/server.py and open the compiled test SWF with the real Adobe Flash Player. You can check what is printed by the terminal running the Python server when you open the test, and replace the Body in output.txt with that.

Then you can run the test in Ruffle with cargo test avm1/netconnection_serialize_arrays and confirm the real Ruffle output matches the expected Flash output that you update output.txt to include.

For AVM2 tests, see https://github.com/ruffle-rs/ruffle/blob/master/CONTRIBUTING.md#apache-flex-sdk

MavenRain added 2 commits May 23, 2026 13:13
  Verifies NetConnection.call serializes genuine arrays as StrictArray,
  fake arrays (Object with numeric keys + length) as Object, and mixed
  arrays as insertion-ordered ECMAArray, against a real Flash Player byte
  capture.

  Co-authored-by: Daniel Jacobs <danielhunterjacobs@gmail.com>

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
  serialize_array bucketed numeric keys into the ECMAArray dense slot and
  the rest into the associative slot, reordering mixed arrays (custom
  string properties interleaved with numeric indices) on the wire.  Real
  Flash emits every enumerable property in insertion order, so mirror
  AVM2's serialize_value heuristic: keys that line up with the enumeration
  index form the dense prefix, and the first key that breaks the run sends
  the remainder to the associative slot.

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
@danielhjacobs
Copy link
Copy Markdown
Contributor

To clarify as I'm unsure if you plan to add more tests and just haven't gotten time this will need tests for the AVM2 side as well, and possibly adjustments to or an additional AVM1 test to make sure the NetConnection AVM1 change is needed in addition to the shared object change, though that's more arguable.

@danielhjacobs
Copy link
Copy Markdown
Contributor

danielhjacobs commented May 25, 2026

Sorry for the duplicate comments I deleted, my cellular data was acting up

@MavenRain
Copy link
Copy Markdown
Author

Roger that, @danielhjacobs . . . will specifically address the AVM2 side shortly

@danielhjacobs
Copy link
Copy Markdown
Contributor

I also still see some issues with the AVM1 side for your logic. Try compiling this test

// --- 1. SETUP THE DATA STRUCTURES ---

var denseArray = new Array();
denseArray.push("dense_0", "dense_1");

var sparseArray = new Array();
sparseArray[0] = "sparse_0";
sparseArray[5] = "sparse_5";

var mixedArray = new Array();
mixedArray.push("mixed_0");
mixedArray["custom_prop"] = "custom_value";

var fakeArray = new Object();
fakeArray["0"] = "fake_0";
fakeArray["length"] = 1;

// NEW: Native Types (Top Level)
var topDate = new Date(1672531200000); // Fixed UTC Timestamp (Jan 1, 2023)
var topXML = new XML("<root><node attr='test'>AVM1</node></root>");

// NEW: Nested Object (Testing recursion fixes for ALL types)
var nestedContainer = new Object();
nestedContainer.deepDense = new Array("deep_0", "deep_1");
nestedContainer.deepSparse = new Array();
nestedContainer.deepSparse[0] = "deep_s0";
nestedContainer.deepSparse[3] = "deep_s3";
nestedContainer.deepDate = new Date(1672531200000);
nestedContainer.deepXML = new XML("<nested>data</nested>");


// --- 2. TEST LOCALCONNECTION (Wire Serialization) ---
var lcReceiver = new LocalConnection();
lcReceiver.onReceiveArrays = function(d, s, m, f, date, xml, n) {
    trace("LC Deserialization Complete: " + d[0] + ", " + s[5] + ", " + date.getTime() + ", " + xml.firstChild.nodeName + ", " + n.deepDate.getTime());
};
lcReceiver.connect("amf0_test_connection");

trace("--- Testing LocalConnection ---");
var lcSender = new LocalConnection();
lcSender.send("amf0_test_connection", "onReceiveArrays", denseArray, sparseArray, mixedArray, fakeArray, topDate, topXML, nestedContainer);


// --- 3. TEST SHAREDOBJECT (Disk Serialization) ---
trace("--- Testing SharedObject ---");
var so = SharedObject.getLocal("avm1_amf_test");
so.data.d = denseArray;
so.data.s = sparseArray;
so.data.m = mixedArray;
so.data.f = fakeArray;
so.data.date = topDate;
so.data.xml = topXML;
so.data.n = nestedContainer;
so.flush();

var soRead = SharedObject.getLocal("avm1_amf_test");
trace("SO Deserialization Complete: " + soRead.data.date.getTime() + ", " + soRead.data.n.deepXML.firstChild.nodeName);


// --- 4. TEST NETCONNECTION (AMF0 Wire Serialization) ---
trace("--- Testing NetConnection ---");
var nc = new NetConnection();
nc.connect("http://localhost:8000/");

var responder = new Object();
responder.onResult = function(res) { trace("NC Result"); };

nc.call("test.avm1", responder, denseArray, sparseArray, mixedArray, fakeArray, topDate, topXML, nestedContainer);

@danielhjacobs
Copy link
Copy Markdown
Contributor

I think this is the expected output.txt for it:

--- Testing LocalConnection ---
--- Testing SharedObject ---
SO Deserialization Complete: 1672531200000, nested
--- Testing NetConnection ---
Navigator::fetch:
  URL: http://localhost:8000/
  Method: POST
  Mime-Type: application/x-amf
  Body: [00, 00, 00, 00, 00, 01, 00, 09, 74, 65, 73, 74, 2E, 61, 76, 6D, 31, 00, 02, 2F, 31, 00, 00, 01, 4E, 0A, 00, 00, 00, 07, 0A, 00, 00, 00, 02, 02, 00, 07, 64, 65, 6E, 73, 65, 5F, 30, 02, 00, 07, 64, 65, 6E, 73, 65, 5F, 31, 0A, 00, 00, 00, 06, 02, 00, 08, 73, 70, 61, 72, 73, 65, 5F, 30, 06, 06, 06, 06, 02, 00, 08, 73, 70, 61, 72, 73, 65, 5F, 35, 08, 00, 00, 00, 01, 00, 01, 30, 02, 00, 07, 6D, 69, 78, 65, 64, 5F, 30, 00, 0B, 63, 75, 73, 74, 6F, 6D, 5F, 70, 72, 6F, 70, 02, 00, 0C, 63, 75, 73, 74, 6F, 6D, 5F, 76, 61, 6C, 75, 65, 00, 00, 09, 03, 00, 01, 30, 02, 00, 06, 66, 61, 6B, 65, 5F, 30, 00, 06, 6C, 65, 6E, 67, 74, 68, 00, 3F, F0, 00, 00, 00, 00, 00, 00, 00, 00, 09, 0B, 42, 78, 56, AA, 0C, 80, 00, 00, 00, 00, 0F, 00, 00, 00, 2A, 3C, 72, 6F, 6F, 74, 3E, 3C, 6E, 6F, 64, 65, 20, 61, 74, 74, 72, 3D, 22, 74, 65, 73, 74, 22, 3E, 41, 56, 4D, 31, 3C, 2F, 6E, 6F, 64, 65, 3E, 3C, 2F, 72, 6F, 6F, 74, 3E, 03, 00, 09, 64, 65, 65, 70, 44, 65, 6E, 73, 65, 0A, 00, 00, 00, 02, 02, 00, 06, 64, 65, 65, 70, 5F, 30, 02, 00, 06, 64, 65, 65, 70, 5F, 31, 00, 0A, 64, 65, 65, 70, 53, 70, 61, 72, 73, 65, 0A, 00, 00, 00, 04, 02, 00, 07, 64, 65, 65, 70, 5F, 73, 30, 06, 06, 02, 00, 07, 64, 65, 65, 70, 5F, 73, 33, 00, 08, 64, 65, 65, 70, 44, 61, 74, 65, 0B, 42, 78, 56, AA, 0C, 80, 00, 00, 00, 00, 00, 07, 64, 65, 65, 70, 58, 4D, 4C, 0F, 00, 00, 00, 15, 3C, 6E, 65, 73, 74, 65, 64, 3E, 64, 61, 74, 61, 3C, 2F, 6E, 65, 73, 74, 65, 64, 3E, 00, 00, 09]
LC Deserialization Complete: dense_0, sparse_5, 1672531200000, root, 1672531200000

@danielhjacobs
Copy link
Copy Markdown
Contributor

Then, for the AVM2 side, I suggest compiling this test:

package {
    import flash.display.Sprite;
    import flash.net.NetConnection;
    import flash.net.Responder;
    import flash.net.LocalConnection;
    import flash.utils.ByteArray;

    public class Test extends Sprite {
        public function Test() {
            
            // --- 1. SETUP THE DATA STRUCTURES ---
            var as3Dense:Array = ["dense_0", "dense_1"];
            
            var as3Sparse:Array = [];
            as3Sparse[0] = "sparse_0";
            as3Sparse[5] = "sparse_5";

            var as3Mixed:Array = ["mixed_0"];
            as3Mixed["custom_prop"] = "custom_value";

            var as3Fake:Object = { "0": "fake_0", "length": 1 };

            // NEW: Native Types (Top Level)
            var as3Date:Date = new Date(1672531200000);
            var as3XML:XML = new XML("<root><node attr='test'>AVM2</node></root>");

            // NEW: Nested Object
            var as3Nested:Object = {
                deepDense: ["deep_0", "deep_1"],
                deepSparse: [],
                deepDate: new Date(1672531200000),
                deepXML: new XML("<nested>data</nested>")
            };
            as3Nested.deepSparse[0] = "deep_s0";
            as3Nested.deepSparse[3] = "deep_s3";


            // --- 2. TEST BYTEARRAY (AMF0 & AMF3 Memory Boundaries) ---
            trace("--- Testing ByteArray AMF0 ---");
            var ba0:ByteArray = new ByteArray();
            ba0.objectEncoding = 0; 
            ba0.writeObject(as3Dense);
            ba0.writeObject(as3Sparse);
            ba0.writeObject(as3Mixed);
            ba0.writeObject(as3Fake);
            ba0.writeObject(as3Date);
            ba0.writeObject(as3XML);
            ba0.writeObject(as3Nested);
            
            ba0.position = 0;
            var readBa0:* = ba0.readObject();
            trace("ByteArray AMF0 Read: " + readBa0[0]);

            trace("--- Testing ByteArray AMF3 ---");
            var ba3:ByteArray = new ByteArray();
            ba3.objectEncoding = 3; 
            ba3.writeObject(as3Dense);
            ba3.writeObject(as3Sparse);
            ba3.writeObject(as3Mixed);
            ba3.writeObject(as3Fake);
            ba3.writeObject(as3Date);
            ba3.writeObject(as3XML);
            ba3.writeObject(as3Nested);
            
            ba3.position = 0;
            var readBa3:* = ba3.readObject();


            // --- 3. TEST LOCALCONNECTION (AMF0 Wire Boundaries) ---
            trace("--- Testing LocalConnection ---");
            var lcReceiver:LocalConnection = new LocalConnection();
            lcReceiver.client = {
                onReceiveArrays: function(d:*, s:*, m:*, f:*, date:*, xml:*, n:*):void {
                    trace("LC Received: " + date.getTime() + ", " + xml.name());
                }
            };
            try { lcReceiver.connect("amf3_test_connection"); } catch (e:Error) {}

            var lcSender:LocalConnection = new LocalConnection();
            lcSender.send("amf3_test_connection", "onReceiveArrays", as3Dense, as3Sparse, as3Mixed, as3Fake, as3Date, as3XML, as3Nested);


            // --- 4. TEST NETCONNECTION (AMF0 & AMF3 Wire Paths) ---
            var responder:Responder = new Responder(
                function(res:Object):void { trace("NC Success"); },
                function(err:Object):void { trace("NC Failed"); }
            );

            trace("--- Testing NetConnection AMF0 ---");
            var nc0:NetConnection = new NetConnection();
            nc0.objectEncoding = 0; 
            nc0.connect("http://localhost:8000/");
            nc0.call("test.avm2.amf0", responder, as3Dense, as3Sparse, as3Mixed, as3Fake, as3Date, as3XML, as3Nested);

            trace("--- Testing NetConnection AMF3 ---");
            var nc3:NetConnection = new NetConnection();
            nc3.objectEncoding = 3; 
            nc3.connect("http://localhost:8000/");
            nc3.call("test.avm2.amf3", responder, as3Dense, as3Sparse, as3Mixed, as3Fake, as3Date, as3XML, as3Nested);
        }
    }
}

Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
@danielhjacobs danielhjacobs requested a review from kjarosh May 28, 2026 02:35
@MavenRain
Copy link
Copy Markdown
Author

@danielhjacobs I agree that #23775 has become the more complete and correct version here. It handles the cases mine doesn't (sparse arrays with holes, top-level XML, nested recursion, and the AMF3 paths), and you can verify the wire bytes against real Flash, which I can't locally. It makes sense to consolidate on #23775 rather than maintain two overlapping fixes, so I'll close this in favor of yours.

I'm happy to help land #23775 however is useful, whether that's review or porting over the AVM2 NetConnection test case if you don't already cover that exact one. Thanks for the thorough diagnosis and the test scaffolding throughout.

@MavenRain MavenRain closed this May 28, 2026
@MavenRain MavenRain deleted the fix/amf-strict-array-for-as3-array branch May 28, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-core Area: Core player, where no other category fits amf Issues relating to AMF serialization/deserialization llm The PR contains mostly LLM-generated code T-fix Type: Bug fix (in something that's supposed to work already)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Genes to Cognition Online - Remaining issues after Flash Remoting support

3 participants